Multiple Kernel Learning for Support Vector Regression ∗

Kernel support vector (SV) regression has successfully been used for prediction of nonlinear and complicated data. However, like other kernel methods such as support vector machine (SVM) classification, the quality of SV regression depends on proper choice of kernel functions and their parameters. Kernel selection for model selection is conventionally performed through repeated cross validation over a range of kernels and their parameters. Multiple kernel learning arises when a range of kernel parameters need to be tuned within one training process, when different types of kernels are applied simultaneously, or when data are from heterogeneous sources and are characterized with different kernels. Multiple kernel learning can improve the efficiency of kernel selection by automatically evaluate the relative importance of the candidate kernels. Inspired by recent developments in kernel selection for SVM classification, we investigate multiple kernel learning for SV regression. Since more slack variables and constraints are involved in the optimization formulation of SV regression than SVM, we can only follow the general procedures used by SVM but cannot directly use the results derived from SVM. We transform the optimization problems of SV regressions using both ε-insensitive loss function and automatic error control into proper forms so that they can be formulated as semidefinite programming (SDP) problems. To avoid the high computational cost of SDP programming, we further formulate multiple kernel SV regression into quadratically constrained quadratic programming optimization problems. Experiments on public and simulated data sets demonstrate that multiple kernel SV regression improves prediction accuracy, reduces the number of support vectors, and helps characterize the propertis of the data.